Thorough testing is one of the most important aspects of software development. In order to test complex software thoroughly, it is not unusual for a software developer to run thousands of tests, some of which might take hours or more to execute. If a software development team utilizes such tests to approve changes to the software, the extensive amount of time required to perform the tests might limit the team's ability to make a code change and have the change be deployed to a production environment in a timely manner. This can be frustrating for the software developers on the software development team.
As a result of the sometimes lengthy time required to perform software tests, some software development teams end up reducing the number of tests that are performed on the software. This means that code changes reach a production environment quicker. However, this also means that the risk that the software includes bugs is greater than if more thorough testing had been performed. A higher risk of deploying software containing bugs is unacceptable in many environments.
One solution to the problems described above is to utilize test platforms that allow for multithreaded execution of tests. Using such a platform, multiple tests can be executed concurrently in the same process. However, this solution often has its own limitations. For example, in order to perform multithreaded execution of tests, the test developer must write the tests in a way that ensures that the tests are safe for multithreaded execution. As a result, creating tests for multithreaded execution might require significant additional work by the software developer. Additionally, multithreaded execution only scales to the limits of the computing resources (e.g. CPU and memory) of the computer upon which they are executed. As a result, the benefits of multithreaded execution of tests might be limited.
The disclosure made herein is presented with respect to these and other considerations.